forked from jesparza/peepdf
-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix object stream parsing #9
Open
michaelweiser
wants to merge
6
commits into
hatching:main
Choose a base branch
from
michaelweiser:object-stream
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Commit 8cc27b6 broke object stream parsing by resetting the content cursor PDFParser.charCounter to zero on every invocation. This broke object stream parsing. Reproducer: $ echo -e "create pdf\ncreate object_stream\nall\nsave /tmp/foo.pdf" | \ peepdf -i Without fix: $ peepdf -j /tmp/foo.pdf Error: An error has occurred while parsing an indirect object!! With this change: JSON output as expected (same for other outputs). $ peepdf -j /tmp/foo.pdf { "peepdf_analysis": { [...] "version": "0.3" } } } Signed-off-by: Michael Weiser <[email protected]>
For unclear reasons, PDFObjectStream.update() delays decoding of the modified raw stream until all references can be resolved. It does however then go on to always try to extract objects from the still empty decoded stream. This produces an error from peepdf cli: $ peepdf image.php Error: An error has occurred while parsing an indirect object!! The error from PDFObjectStream.update() is "Missing offsets in object stream" because self.decodedStream is still empty at that point, making offsetsSection and eventually the numbers list empty, causing the abort. This is triggered by /Length being a reference and setting updateNeeded to True. Sample: https://www.infotek.co.jp/pdflib/demo/sample/image.php. Relevant PDF structure: 32 0 obj <</Length 43 0 R/Filter/FlateDecode/Type/ObjStm/N 7/First 47>> stream [...] endstream endobj 43 0 obj 461 endobj (Length in dict of object 32 R-eferences object 43 which contains 461 what presumably is the length of the stream - which does not seem to be used or checked for consistency by peepdf atm, btw.) This resolves the first half of jesparza#70 in that force mode is no longer necessary to parse such files at all. Signed-off-by: Michael Weiser <[email protected]>
With the previous change deferring reading of objects from the decoded stream until references can be resolved, it now runs into jesparza#70. This change provides a different approach in fixing it to hatching#6 by syncing it with the other locations where the identical code is in use: 1. Force the numbers extracted by re.findall to int() as before, avoiding the TypeError exception: Traceback (most recent call last): File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/main.py", line 409, in main ret, pdf = pdfParser.parse(fileName, options.isForceMode, options.isLooseMode, options.isManualAnalysis) File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 7117, in parse ret = body.updateObjects() File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 4291, in updateObjects object.resolveReferences() File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 3256, in resolveReferences ret = PDFParser.readObject(objectsSection[offset:]) TypeError: slice indices must be integers or None or have an __index__ method 2. Instantiate a new PDFParser object by adding the missing braces, avoiding another TypeError because readObject is no class method: Traceback (most recent call last): File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/main.py", line 409, in main ret, pdf = pdfParser.parse(fileName, options.isForceMode, options.isLooseMode, options.isManualAnalysis) File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 7118, in parse ret = body.updateObjects() File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 4292, in updateObjects object.resolveReferences() File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 3256, in resolveReferences ret = PDFParser.readObject(objectsSection[offset:]) TypeError: unbound method readObject() must be called with PDFParser instance as first argument (got str instance instead) 3. Explicitly force the id to be an int() as well and append it do the list of indices as at the other callsites of this code. This solves no issue I have run into but seems sensible to avoid other potential TypeErrors and keep internal bookkeeping of the object consistent. This should conclusively resolve jesparza#70 and supersedes hatching#6. Signed-off-by: Michael Weiser <[email protected]>
michaelweiser
force-pushed
the
object-stream
branch
from
March 6, 2020 18:53
cb585df
to
431e872
Compare
This was referenced Mar 6, 2020
In PDF files the Cross-Reference Table or a Cross-Reference Stream contain byte-offsets for the start of objects within the file or the uncompressed stream. Such an offset does not always point the first byte of the initial token (see ISO 32000-2008 section 7.2.2) of the referenced object. The object may be preceded by white-space characters and comments. Without this commit PDFParser.readSymbol() fails to read a symbol, if the first character to be processed is a white-space character. This commit changes PDFParser.readSymbol() to skip leading white-space characters. (PDFParser.readSymbol() already skips any number of leading comments followed by white-space characters.) This enables passing of PDF-files with sloppy cross reference offsets.
An object of class PDFArray can contain JS-code, if one or more array-elements contain JS-code. The getter method was simply missing.
Pushed some more fixes to object-stream-related problems, particularly one containing javascript or sloppy object reference offsets. |
A previous commit adjusted readSymbol() to skip leading whitespace in order to avoid errors with sloppy cross references. This did not fix handling of literals such as numbers and booleans in readObject() because they're not accessed using readSymbol(). Also, adjusting the very low-level readSymbol() function might generate fallout. So instead, this change moves the skipping of leading whitespace into readObject() so that it affects all types of referenced objects equally but not all symbol lookups altogether. Signed-off-by: Michael Weiser <[email protected]>
michaelweiser
force-pushed
the
object-stream
branch
from
July 2, 2020 14:04
5a5887a
to
90720a4
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Commit 8cc27b6 broke object stream parsing by resetting the content
cursor
PDFParser.charCounter
to zero on every invocation. Unfortunately it's unclear from the change what this was fixing at the time but it certainly broke object stream parsing. Reproducer:Without fix:
With this change: JSON output as expected (same for other outputs).
Given the object created by above call in local variable
content
:at https://github.com/jbremer/peepdf/blob/11dab2ead9bf344e6aa8cca7f82a17771350f1be/peepdf/PDFCore.py#L7873
self.charCounter
points to the end of the dictionary (>>
) and is 65.readUntilSymbol()
is supposed to advance it to thestream
keyword which is three characters away. It first determines the substring to search based on the current value of65
, then resets the counter to zero, finds the symbol three characters into the string and finally setsself.charCounter
to3
. The following calls toreadSymbol()
and colleagues fail silently but advancecharCounter
beyond the next newline. From there the next call toreadUntilSymbol('endstream')
tries to extract the actual objectstream content. BecausecharCounter
is still in the middle of the dictionary, it extracts part of that as well and decoding of the object stream eventually fails.Some poor man's debug prints showcase the issue:
Output without the fix:
Output with the fix: